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ABSTRACT 

A  discussion  of  convolutional  encoding  and  sequential 
decoding  is  given  as  background  for  the  experimental  phase 
of  this  research.   A  modified  stack  algorithm  is  programmed 
to  simulate  a  sequential  decoder  for  use  in  the  study  of 
time-varying  convolutional  codes. 

The  decoder  simulator  is  used  to  analyze  the  structure 
of  codes  with  the  aim  of  finding  a  systematic  method  by 
which  to  construct  time-varying  convolutional  codes  with 
good  decoding  properties.   Some  difficulties  in  construction 
techniques  are  discussed. 

Extensive  computer  simulation  comparing  decoding  perfor- 
mance of  time-varying  codes  to  that  of  fixed  codes  Is 
reported.   The  conclusion  of  these  comparisons  is  that  the 
time-varying  codes  studied  thus  far  are  of  theoretical 
interest  only. 
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I.   INTRODUCTION 

A.   BACKGROUND 

Since  noise  is  always  present  in  electrical  communica- 
tions circuits,  there  is  a  non-zero  probability  that  trans- 
mitted signals  will  be  altered  causing  errors  at  the 
receiver.   When  digital  information  is  transmitted  directly 
from  machine  to  machine,  erroneous  reception  of  even  a 
single  bit  of  information  may  change  the  complete  meaning 
of  a  message.   It  is  for  this  reason  that  error-correcting 
codes  have  been  devised.   These  codes  should  provide  a  means 
of  communicating  with  any  required  degree  of  reliability, 
and  should  be  easily  implemented  with  existing  digital 
equipment . 

Shannon  [1]  has  shown  that  codes  meeting  the  first  cri- 
terion do  exist  provided  that  the  data  rate  does  not  exceed 
some  maximum,  called  channel  capacity.   The  statistical 
characteristics  of  the  space  channel  have  indicated  that 
the  use  of  convolutional  codes  with  sequential  decoding  is 
one  of  the  best  methods  of  attaining  the  reliability  required 
over  this  type  of  channel.   In  this  chapter  a  general  con- 
volutional encoder  and  sequential  decoding  are  briefly 
discussed.   A  description  of  the  channel  model  used  in 
simulation  studies  is  also  given. 


B.   CONVOLUTION AL  CODES 

Forney  [2]  has  given  the  following  definition:   "An 
(n,k)  convolutional  encoder  over  a  finite  field,  F,  is  a 
k-input  n-output  constant  linear  causal  finite-state  sequen- 
tial circuit."   The  rate  of  a  general  convolutional  encoder 
is  defined  to  be  k/n . 

A  general  binary  rate  H   convolutional  encoder  of  memory 
m  is  shown  in  Figure  1.   The  input,  {x  },  is  a  binary 
sequence,  called  the  information  sequence,  and  the  outputs 
are  two  binary  sequences,  {y,   }  and  {y,   },  either  commu- 
tated  to  form  a  single  sequence  or  transmitted  independently 
At  each  unit  of  time  the  encoder  transforms  one  input  digit 
into  a  sequence  of  two  output  digits.   It  can  be  seen  from 
Figure  1  that  the  two  output  digits  depend  on  the  present 
information  digit  and  the  m  previous  information  digits. 

The  transform  A(D)  of  the  sequence  a~  ,a-,  ,a?  ,  .  .  .  is 
defined  as  a  formal  power  series  in  the  indeterminate,  D. 

A(D)  =  aQ  +  a1D1  +  a2D2  +  ... 

If  the  sequence  afi ,a, ,a? , . . .  has  only  a  finite  number  of 
non-zero  elements  then  A(D)  is  a  polynomial  in  D.   The 
D-transforms  of  the  sequences  g^   ,g-|   ,g2   j--->gm  > 


G 


(Dr^m 


i(D)    -g(D    +    g(DD   +    gCDD2    +    ...    +g(DD 
G2(D)    -    g<2>    +    g^D   +    g<2V    +    ...    +    g^D™, 

are  called  the  code  generating  polynomials  of  the  encoder 
In  Figure  1,  if  g£    =  0 ,  no  wire  is  present  and,  if 


® — °y 


(i) 


-<+>— 


e 


-  Unit  Time  Delay  Element 


-  Constant  Multiplier 


-  Modulo-2  Adder 


Figure  1.   General  Binary  Rate  H   Convolutions!  Encoder 
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gk   -  1,  a  wire  is  present.   If  the  generating  polynomials 
are  time-invariant,  the  encoder  is  known  as  a  fixed  convolu- 
tional  encoder. 

One  approach  to  looking  at  the  encoder  is  noting  the 
outputs  at  time  k. 

yU;  =   z   g  Jjx,_.  ,     j  =  1,2   (modulo-2  sum) 
i  =  0 

This  is  the  convolution  of  the  sequence 


'0       igl       ,,,,,gm 


with  the  sequence 


X„  >  XT  )  xo  > 


o»  Is    2ym  '  ' 


■> 


hence  the  name  "convolutional"  code. 


(J) 


The  D-transform,  Y.(d),  of  the  sequence  {y_/. J  }  i: 


given  by 

Y  (D)  =  G  (D)X(D)  ,      j  =  1,2 
j        j 

where  X(D)  is  the  transform  of  the  input  sequence.   There- 
fore, as  in  continuous  linear  systems,  convolution  in  the 
time  domain  corresponds  to  multiplication  in  the  D-transform 
domain,  where,  for  binary  systems,  the  multiplication  is 
performed  modulo-2. 

Another  convenient  way  to  view  convolutional  encoders 
is  through  a  matrix  description.   Using  Forney's  stipulation 
[2]  that  a  convolutional  encoder  is  a  causal  linear  circuit, 
it  can  be  assumed  that  the  information  sequences  begin  at 
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time  zero.   Therefore,  for  a  rate  h   fixed  convolutional 
encoder,  the  information  sequence  is 


—      |^Q>-X-]>-X'P»  *  *  *  I    S 


and  the  corresponding  output  sequence  is 


Y  = 


(1)  (2)  „(1)„(2)  „(1)  (2) 


'yl   yl 


,y. 


The  encoding  equations  for  the  encoder  can  be  written 


Y  =  X  G, 


where  for  an  encoder  with  memory  m, 


G  = 


G 
— o 

°1 

°2 

GOO 

— m        —          — 

0 

^0 

°1 

G      ,    G           0 
— m-1   — m        — 

0 

0 

°0 

•     •     • 

G          ^,      G          n       G 

— m-2   —m-1    — m 

is  called  the  generator  matrix  and 


G.  = 
— l 


(1)   (2) 
gi   gi 


0  <  i  < 


Encoders  utilizing  generator  polynomials  which  vary 
with  time,  u,  are  known  as  time-varying  convolutional 
encoders.   If  G.  (u)  =  G.  (u  +  T)  ,  0  <_  u  <  °°,  the  encoder  is 
a  periodic  time-varying  convolutional  encoder  with  period  T. 
The  generator  matrix  for  a  time-varying,  rate  \   encoder  with 
memory  m  is  given  by 
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G  = 


GQ(0)  0^(1)  G2(2) 
0    G0(l)  Gx(2) 


G  (m)       0         0 
— m         —        — 

^m-l(rn)   ^(m+1)      0 


0    i0(2)  ••*  ^m-2(m)  ^m-l(m+1)  ^-m(m+2) 


where  G.(u)  =  fg|1)  (u)  g|2)(u)l. 


Costello  [31 »  P-  81,  has  given  a  theorem  which  indicates 
that  there  exists  at  least  one  non-systematic ,  periodic, 
time-varying  code  with  decoding  properties  which  may  be 
vastly  better  than  those  of  fixed  codes.   The  theorem, 
which  will  be  further  discussed  in  Chapter  II,  is  the  major 
factor  which  prompted  the  search  for  such  a  code  in  this 
research  project. 

C.   TREE  STRUCTURE  OP  C0NV0LUTI0NAL  CODES 

Now  consider  the  rate  h,    memory-2  encoder  in  Figure  2. 
The  delay  units  can  be  a  two-stage  digital  register.   In 
this  case 

G1(D)  =  1 

G2(D)  =  1  +  D  +  D2 

and  the  code  generated  is  called  systematic .   This  term  is 
used  for  codes  which  have  the  property  that  the  information 
sequence  appears  explicitly  in  the  output.   Had  G^(D)  con- 
tained other  non-zero  coefficients  of  powers  of  D,  the  code 
would  have  been  non-systematic . 

To  see  how  the  convolutional  encoder  constructs  the 
coded  output  codeword,  initially  set  all  stages  of  the 
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register  to  zero.   Sequentially  shift  information  bits  into 
the  register  and  after  each  shift,  observe  the  outputs  y, 
and  y« .   For  example,  the  information  sequence  1  0  1  1  ... 
will  yield  the  encoded  sequence  11  01  10  10  ...  as  the 
transmitted  symbols. 

From  this  example  it  can  be  seen  that  the  transmitted 
symbols  depend  on  the  present  and  two  past  information  bits. 
The  structure  of  the  set  of  all  output  codewords  can  be 
seen  from  a  code  tree ,  constructed  as  follows.   For  an 
information  sequence ,  X  =  [x~  ,x,  ,  . . .  3x,  -,],  the  set  of  all 
2"J  L-component  input  vectors  is  diagrammed  on  an  input  tree 
using  the  convention  that  the  upper  branch  diverging  from 
a  node  corresponds  to  shifting  a  zero  into  the  encoder 
register  and  the  lower  branch  to  shifting  in  a  one.   The 
code  tree  is  completed  by  adding  to  each  branch  of  the  input 
tree  the  output  digits  associated  with  it.   Using  the  example 
given  previously,  the  tree  of  Figure  3  is  constructed.   It 
can  be  seen  that  each  possible  information  sequence  defines 
a  unique  path  through  the  tree.   The  encoded  sequence  of 
the  example  is  shown  by  the  bold  path  in  Figure  3-   The 
concept  of  the  tree  structure  of  convolutional  codes  leads 
directly  to  a  discussion  of  sequential  decoding. 

D.   SEQUENTIAL  DECODING 

Sequential  decoding,  introduced  by  VTozencraft  [H]  ,  is 
probabilistic  in  nature.   The  decoder  has  available,  or  is 
able  to  calculate,  the  channel  error  probability  character- 
istics and  a  copy  of  the  code  tree  used  by  the  encoder. 
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Fieure  2.   Binary  Rate  \   Encoder  of  Memory  Two. 
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Code  Tree  for  the  Encoder  of  Figure  2 
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Sequential  decoding  algorithms  cannot,  in  practice,  con- 
struct the  complete  tree  for  each  received  sequence  since 
the  number  of  nodes  grows  exponentially  with  the  length  of 
the  sequence.   The  decoder  in  actuality  maintains  a  replica 
of  the  convolutional  encoder.   Wozencraft  and  Jacobs  [51, 
p.  ^26,  noted  that  the  input,  X,  to  the  encoder  may  be 
regarded  as  a  set  of  instructions  which  direct  the  encoder 
on  a  unique  path  through  the  code  tree.   Consider  a  binary 
symmetric  channel  (BSC)  and  a  rate  \   encoder.   The  decoder, 
with  a  replica  of  the  encoder,  starts  at  the  first  node  in 
the  code  tree,  generates  both  succeeding  branches  and  fol- 
lows the  one  that  agrees  most  closely  with  the  received 
sequence.   The  process  continues  until  the  decoder  reaches 
one  of  the  2   terminal  nodes  for  an  L-bit  information 
sequence.   At  this  point  the  sequence  is  decoded.   In  the 
case  when  the  channel  is  noiseless,  the  result  is  trivial, 
provided  that  both  branches  diverging  from  any  one  node 
differ  in  at  least  one  bit.   Such  a  difference  may  be 
guaranteed  by  ensuring  that  g!:    =  1 ,  as  can  be  seen  from 
Figure  2  and  the  resulting  code  tree  of  Figure  3. 

When  noise  is  introduced  over  the  channel,  the  foregoing 
process  is  not  sufficient  to  decode  a  message.   In  this 
case,  if  neither  branch  emanating  from  a  node  corresponds 
in  all  places  to  the  respective  received  bits,  the  decoder 
follows  whichever  branch  agrees  best.   If  the  decoder  makes 
an  error  at  a  particular  node,  it  is  unlikely  to  find  a 
path  along  subsequent  branches  which  agrees  well  with  the 
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remainder  of  the  received  sequence.   At  this  point  the 
decoder  is  programmed  to  go  back  to  the  last  "best"  node 
and  search  another  path.   In  practical  decoding  algorithms, 
a  metric  value  is  assigned  to  every  branch  in  the  tree.   If 

I  =  [yi>y2'---yn] 

are  the  code  symbols  on  the  branch  and 
R  =  [r^,...,^] 

are  the  corresponding  received  symbols,  then  the  branch 
metric  is  taken  to  be 


a  = 


n 

Z 

i  =  l 


HrJy.} 

Log  — /  v1 B 


(1.1) 


where  P{r.  j  y . }  is  the  channel  transition  probability  func- 
tion, p(r.)  is  the  nominal  received  symbol  probability 
density  function  and  B  is  a  bias  term.   Proper  choice  of  B 
will  ensure  that,  on  the  average,  metric  values  along  the 
correct  path  are  positive  while  those  along  the  incorrect 
path  are  negative.   Nodes  are  assigned  values  equal  to  the 
sum  of  the  branch  metrics  along  the  path  leading  to  the 
node.   The  decoder  need  only  look  for  a  path  with  generally 
increasing  value  to  find  the  correct  path. 

C,  the  number  of  computations  performed  by  a  sequential 
decoder,  can  be  defined  as  the  number  of  nodes  examined  in 
the  search  for  a  path  through  the  tree.   C  is  a  random 
variable,  since  the  number  of  computations  performed 
depends  upon  the  received  sequence,  which  in  turn  depends 
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upon  the  channel  noise.   Jacobs  and  Berlekamp  [6]  have  shown 
that  the  probability  distribution  function  of  C  behaves  as 

P{C  >  N}  >  KN~P 

for  large  N,  where  p  is  a  number  depending  only  on  the 
channel  and  the  rate  of  the  code,  and  K  varies  slowly  with 
N.   The  distribution  is  termed  as  Paretian.   This  unfor- 
tunate behavior  of  the  distribution  of  computation  seems  to 
be  the  major  drawback  to  the  use  of  sequential  decoders. 
The  implication  is  that  decoding  speed  is  a  variable  and 
received  data  must  be  stored  in  a  buffer  as  received  until 
it  can  be  processed  by  the  decoder.   The  Pareto  distribution 
of  computations  indicates  that  there  will  be  times  when  the 
decoder  will  fall  so  far  behind  that  the  buffer  will  over- 
flow.  In  these  cases  there  are  two  options: 

(1)  Divide  the  information  sequence  into  "frames." 
When  the  buffer  overflows  during  a  frame,  put  the 
frame  aside  to  be  processed  later  and  let  the 
decoder  start  the  next  new  frame. 

(2)  If  overflow  occurs,  request  that  the  transmitter 
stop,  resynchronize ,  and  begin  sending  again. 

It  can  be  verified  that  C  <  °°,  if  and  only  if  p  >  1. 
The  code  rate  such  that  p  =  1  is  called  R     •   The  signifi- 
cance of  this  is  that  for  rates-  R  <  Rcom  ,    C  Is  bounded, 

while  for  R  >  R     ,  C  =  °°.   Therefore,  decoding  is  con- 

comp  ' 

siderably  more  difficult  at  rates  substantially  above 

R     .   Usually,  in  sequential  decoding,  RrtrtTn„  is  greater 
comp  comp 

than  or  equal  to  one-half  channel  capacity. 

One  major  advantage  to  sequential  decoding  schemes, 
exemplified  by  Gallager  [7],  p.  284,  is  the  fact  that  the 


probability  of  error  decays  exponentially  with  encoder 
memory  m;  thus  reliability  can  be  increased  as  much  as 
desired  with  relatively  small  increase  in  encoder  complex- 
ity, and  almost  no  change  in  decoder  complexity.   Another 
advantage  is  the  relative  ease  of  implementation  of  sequen- 
tial decoding  systems  versus  other  systems  with  comparable 
reliability . 

These  advantages  of  sequential  decoding  weigh  so  heavily 
that,  in  spite  of  the  necessity  of  operating  at  rates  sig- 
nificantly below  capacity,  it  is  generally  recognized  that 
systems  employing  convolutional  codes  and  sequential  decod- 
ing are  the  most  practical  means  of  achieving  high- 
reliability,  low  power  communication  over  memoryless 
channels . 

E.   BINARY  SYMMETRIC  CHANNEL 

Throughout  the  experimental  phase  of  this  research  pro- 
ject, the  binary  symmetric  channel  was  used  as  the  channel 
model,  and  it  is  shown  in  Figure  4  for  completeness.   With, 
probability  q  =  1  -  p,  the  output  symbol  is  a  replica  of 
the  input  symbol,  and  with  probability  p,  it  is  the  comple- 
ment of  the  input.   Those  crossover  probabilities,  p,  with 

corresponding  R     ,  channel  capacity,  and  p  utilized  in 
^      G   comp 3 

simulation  are  listed  in  Table  I.   With  R  =  h,    these  choices 

of  p  represent  operation  at  approximately  RcomD  and  ten 

percent  below  R  _  . 
^  comp 
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Figure  4.   Binary  Symmetric  Channel 
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0.03125 
0.04688 


0.562 
0.491 


0.793 
0.727 


1.371 
0.950 
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II.   DISTANCE  DEFINITIONS  AMD  BOUNDS 

The  distance  between  codewords  in  a  convolutional  code 
is  one  measure  used  in  analyzing  the  error-correcting  capa- 
bilities of  the  code.   In  this  chapter,  several  different 
distance  measures  will  be  defined  for  convolutional  encoders 
and  some  established  lower  bounds  on  these  measures  will 
be  shown. 

A.   DISTANCE  MEASURES 

1 .   Minimum  Distance 

Massey  [8],  p.  15,  has  defined  the  minimum  distance, 

d  .  ,  of  a  convolutional  code  to  be  the  smallest  number  of 
mm' 

symbols  in  which  two  initial  codewords  differ  that  do  not 
have  identical  sets  of  first  information  symbols.   That  is, 

d  .   =   min   du  ( [X  G]  ,  [X'G]  )  , 
mm    x    ,„f   H m' m   ' 

— o  — o 

where  dH(Z,Z')  denotes  the  Hamming  distance  between  the  two 

arguments,  [Z_]   means  the  first  (m+1)  blocks  of  the  sequence 

Z,  and  m  is  the  memory  of  the  encoder.   Generalized  to  time- 
varying  encoders,  this  becomes 

dmin  "  J*"     J*%,    dH  ([*  2W  [*'aW- 

0  <  u  <  °°   X  ?X' 
—       — u  — u 

If  the  code  is  periodic  with  period  T,  the  definition  reduces 
to  the  following: 
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d    .       =        min  min    du    ( [X   G]     .     ,     [X'G]     .     ). 

min        0<u<T      X   *X»    H  "  u+m  "  u+m 

—       — u  — u 

Minimum  distance,  so  defined,  is  also  known  as  Feedback 
Decoding  Minimum  Distance.   The  term  "feedback"  derives  from 
the  fact  that  in  some  schemes  of  decoding,  error  propagation 
occurs  as  a  result  of  internal  feedback  in  the  decoder. 
That  is,  decoded  versions  of  prior  sequences  are  used  to 
decode  future  sequences  and  earlier  decoding  errors  can 
affect  future  decoding.   On  the  other  hand,  decoding  methods 
having  no  internal  feedback  have  been  termed  definite 
decoding  by  Robinson  [9] • 

Costello  [3],  P-  25,  has  riven'  an  argument  to  show  that, 
insofar  as  distance  properties  are  concerned,  it  is  not 
necessary  to  consider  non-periodic  time-varying  encoders. 
Throughout  the  rest  of  this  chapter,  all  time-varying 
encoders  will,  therefore,  be  considered  to  be  periodic. 

2 .   Order-j  Column  Distance 

If  the  encoder  input  is  a  binary  sequence  beginning 

with  a  "1",  and  the  encoder  output  is  observed  up  until 

time  j,  the  order-j  column  distance,  d.,  is  the  minimum 

weight  of  all  such  output  sequences.   For  a  periodic 

encoder,  d.  is  given  by 
J 

d.  =   min     min  d   ([X  G]      [X»G]   ,) 
J    0  <u<T   X  *X«      -  -  u+j    -   u+j 
—       — u  — u 

for  j  =  0,1,2,...,  and  dro  =  Dim  d.  .   From  the  foregoing 

i  -voo    <J 

0 

definitions  of  minimum,  distance,  it  can  be  seen  that 

d  .   =  d  . 
min    m 
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3 .   Free  Distance 

In  a  convolutional  encoder  of  memory  m,  any  partic- 
ular information  symbol  can  affect  the  output  sequences 
over,  at  most,  (m+1)  time  units.   During  this  time,  for  an 
(n,k)  encoder,  a  total  of  (m+l)n  symbols  are  transmitted. 
The  code  is  said  to  have  a  feedback  decoding  constraint 
length ,  n.  =  (m+l)n  symbols.   Since  sequential  decoders  are 
not  limited  to  consider  only  one  constraint  length  of 
received  digits  while  attempting  to  decode  a  particular 
block  of  transmitted  digits,  a  more  meaningful  distance 
measure  called  free  distance  has  become  used.   The  free 
distance  is  defined  over  the  entire  encoded  sequence,  and 
thus  is  a  more  appropriate  distance  measure  for  a  decoder 
which,  considers  the  entire  rec-eived  sequence  in  making  a 
decision  as  to  which  symbols  were  transmitted. 

Free  distance  has  been  defined  as  the  minimum  weight 
encoded  sequence  such  that  X  ^  0 . 

d-    =  min  W„  (X  G) 
free    ^Q   H   -  - 

where  W^CZ)  denotes  the  Hamming  weight  of  the  argument.   It 

can  be  shown  that  d~    =  Lim  d..   Since  free  distance  is  a 

free    .     l 
x  L  cc    /J-*"00 

property  of  the  code  itself,  it  appears  that  this  is  the 
distance  measure  of  concern  when  considering  time-varying 
codes  . 

B.   BOUNDS  ON  DISTANCE  MEASURES 

It  is  desirable  from  an  error-correcting  capability 
standpoint  to  maximize  the  particular  distance  measure 
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with  which  one  is  concerned.   Although  it  is  not  always 
clear  how  to  construct  codes  with  good  distance  properties, 
it  is  possible  to  calculate  bounds  on  these  distance 
measures . 

1 .  Feedback  Decoding  Minimum  Distance  for  Fixed  Codes 
Wozencraft  and  Reiffen  [10],  p.  51,  and  Massey  [8], 

p.  15,  have  shown  that  there  exists  at  least  one  fixed 
binary  convolutional  code  with  a  given  rate  R,  and  con- 
straint length  n..  such  that  d  .   is  the  largest  integer 
to    A'  mm 

which  satisfies 

H  1^1  i  1  -  R  , 

where  H(x)  =  -  [xLog-x  +  (l-x)Logp(l-x)]  is  the  binary 
entropy  function.   Since  fixed  codes  can  be  considered  to 
be  periodic  codes  with  period  one,  this  lower  bound  also 
applies  to  the  class  of  periodic  codes.   This  bound  is 
asymptotically  the  same  as  Gilbert's  lower  bound  on 
minimum  distance  for  block  codes  with  the  same  rate  and 
constraint  length   [11] .   The  Gilbert  bound  is  a  universally 
accepted  criterion  of  comparison  for  codes. 

2 .  Definite  Decoding  Minimum  Distance  for 
Systematic  Periodic  Codes 

Wagner  [12],  has  shown  that  there  exists  at  least 

one  binary  convolutional  code  with  period  T  =  m+1 ,  and  rate 
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where  d   and  n   are,  respectively,  different  definitions  of 
w       w 

definite  decoding  minimum  distance  and  definite  decoding 
constraint  length  given  by  Robinson  [91 .   This  bound  is 
only  satisfied  for  R  <_  \ .   Wagner  was  the  first  to  derive 
a  Gilbert  bound  for  this  class  of  codes  and  proved  that  the 
bound  holds  for  a  class  of  systematic  codes  of  period 
T  =  (3m+l)/2  if  m  is  odd  or  T  =  3m+l  if  m  is  even.   This 
was  the  first  class  of  codes  investigated  in  this  project 
and  will  be  further  discussed  later.   This  bound  has  been 
extended  to  all  rates  by  Costello  and  Morrissey  [13]  to  the 
following:   There  exists  at  least  one  systematic,  periodic, 
time-varying  convolutional  code  such  that 

TTfdDD)   >    lzR 

KdJ  - 1+R  5 

where  d^  and  nnn  are  Robinson's  definite  decoding  parame- 
ters . 

3 .   Free  Distance  for  Non-systematic  Periodic  Codes 
An  obvious  lower  bound  on  free  distance  can  be 
obtained  using  the  following  facts  concerning  minimum 
distance  and  column  distance: 

d  .   =  d   :  d„     =  Lim  d.  :  d.,,  >  d. 
mm    m    free    ..     j     j+1  -  j 

Therefore,  d  .   <  d„    and  all  lower  bounds  on  d  .   are 
5   mm  —   free  mm 

also  lower  bounds  on  d„ 

free 

Costello  [31,  p.  8l,  has  proved  a  lower  bound  on 
free  distance  for  non-systematic,  periodic  codes  which  is 
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much  tighter  than  corresponding  bounds  for  fixed  codes.   His 
result  states  that  there  exists  at  least  one  non-systematic 
periodic  code  such  that 

Lim  ^e     "(I/'1) . 

m->°°    nA      H(l-2    )  -  R  -  1 

This  lower  bound  is  very  close  to  the  generalized  McEliece 
and  Rumsey  upper  bound  for  non-systematic  fixed  codes  [31  s 
p.  95-   The  tightness  of  this  lower  bound  and  its  closeness 
to  the  upper  bound  for  fixed  codes  suggests  the  existence 
of  periodic  codes  which  achieve  a  much  lower  probability  of 
decoding  error  than  corresponding  fixed  codes.   It  is  of 
interest,  therefore,  to  find  such  codes  and  to  determine 
if  their  superiority  over  fixed  codes  is  sufficiently  great 
to  justify  the  additional  complexity  of  a  time-varying 
encoder. 
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III.   THE  SIMULATED  SEQUENTIAL  DECODER 

Sequential  decoding,  as  discussed  in  Chapter  I,  is  a 
method  for  decoding  information  which  has  been  encoded, 
using  tree  codes,  for  transmission  over  a  noisy  channel. 
Several  sequential  decoding  algorithms  have  been  introduced 
and  tested.   In  this  chapter  a  detailed  description  ofthe 
algorithm  used  in  this  work  will  be  given  along  with  an 
explanation  of  the  use  of  the  algorithm  in  an  attempt  to 
construct  good  time-varying  codes. 

A.   THE  DECODING  ALGORITHM 

The  algorithm  used,  suggested  in  independent  work  by 
Zigangirov  [14]  and  Jelinek  [15] ,  is  simple  to  describe 
and,  with  modifications  proposed  by  Jelinek,  is  faster  than 
other  competing  algorithms.   In  order  to  attain  the  speed 
advantage  without  an  increase  in  error  probability,  the 
memory  required  for  use  by  the  decoder  must  be  increased 
considerably.   A  description  of  the  basic  algorithm  without 
modifications  follows. 

As  stated  in  Section  I.D,  metric  values  are  assigned  to 
each  branch  in  the  code  tree.   This  metric  value,  given  by 
equation  (1.1),  is  a  branch  likelihood  function.   The  sum 
of  the  branch  likelihood  functions  on  a  path  leading  to  a 
particular  node  is  called  the  node  likelihood  value.   The 
decoder  consists  of  a  portion  of  computer  memory  called 
the  stack  which  is  an  ordered  list  of  nodes.   The  stack  is 
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arranged  in  decreasing  order  of  node  likelihood  values  with 
the  highest  node  having  the  greatest  value  of  all  nodes  in 
the  stack.   The  stack  is  first  loaded  with  the  value  of  the 
origin  node  which  is,  thus,  the  "top"  node  in  the  stack. 
The  decoder  then  proceeds  as  follows : 

(1)  Compute  the  likelihood  values  of  the  two  successors 
to  the  top  node  and  place  them  into  the  stack  in 
order  determined  by  their  values. 

(2)  Eliminate  the  node  whose  two  successors  were  just 
added. 

(3)  If  the  new  "top"  node  is  a  terminal  node  in  the 
tree,  stop.   Otherwise  go  to  (1). 

In  other  words,  after  the  k    step,  the  stack  will  contain 
exactly  k+1  likelihoods  corresponding  to  paths  of  various 
lengths  and  different  end  nodes.   On  the  (k+1)    step,  the 
decoder  searches  the  stack  to_  find  the  largest  stored  like- 
lihood, determines  the  path  to  which  it  corresponds,  and 
replaces  that  likelihood  with  the  likelihoods  of  the  two 
successor  nodes.   When  the  process  halts,  the  top  node  in 
the  stack  determines  a  path  from  the  origin  node  to  a 
terminal  node  in  the  tree. 

Geist  [16],  p.  38,  has  shown  that  the  Zigangirov- 
Jelinek  algorithm  chooses  a  path  sQ,s,  , .  . .  ,s,.  through  the 
tree  which  satisfies  the  following  conditions: 

(1)  s„  is  the  origin  node. 

(2)  The  branch  leading  from  any  particular  node,  s^3 

to  a  node  in  the  next  level  of  the  tree  is  the  first 
branch  of  one  of  the  paths  from  s^  to  the  end  of 
the  tree  having  the  greatest  minimum  likelihood 
value . 

In  essence,  the  result  of  decoding  is  such  that  the 

decoder  picks  a  node,  checks  all  paths  from  that  node  to 
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the  end  of  the  tree  to  find  the  one  with  greatest  minimum 
value,  and  "moves"  one  node  farther  along  that  path.   The 
process  repeats  until  a  terminal  node  is  reached.   To  meet 
the  essential  feature  of  sequential  decoding  given  by  Jacobs 
and  Berlekamp  [6] ,  the  branches  must  be  examined  sequen- 
tially, so  that  at  any  node  of  the  tree  the  decoder's  choice 
among  a  set  of  previously  unexplored  branches  may  not  depend 
on  received  branches  deeper  in  the  tree.   Therefore,  the 
decoder  does  not  in  actuality  look  ahead  on  all  possible 
paths,  but  selects  for  a  node  to  be  extended,  the  one  with 
greatest  likelihood  value  from  the  set  of  all  nodes  in  the 
stack . 

Since  the  decoder  performs  one  computation  for  each  node 
it  examines,  and  with  non-zero  probability  the  number  of 
nodes  examined  grows  quite  large,  there  is  a  possibility  of 
stack  overflow.   In  simulation  studies  this  was  handled  by 
a  cessation  of  decoding  of  the  particular  frame  during  which 
overflow  occurred  and  starting  the  next  frame.   The  frame 
which  caused  overflow  was  then  counted  as  an  erasure. 

A  problem  with  the  Zigangirov-Jelinek  algorithm  is  the 
excessive  time  required  to  keep  the  stack  ordered  so  that 
the  nodes  are  in  decreasing  order  of  likelihood  values. 
This  problem  is  alleviated  when  the  algorithm  is  modified 
as  proposed  by  Jelinek  [15].   The  modified  version,  now  to 
be  discussed,  is  the  one  which  was  used  in  the  experimental 
phase  of  this  work. 

The  modification  consists  of  quantizing  the  likelihood 
values  and  placing  nodes  in  ordered  bins  according  to  their 
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values.   The  stack  is  then  arranged  as  random  access  storage 
with  descriptions  of  each  node  examined  placed  into  the 
stack  sequentially  from  the  top.   Decoding  proceeds  as 
follows : 

(1)  Select  a  node  from  the  highest  non-empty  bin,  com- 
pute the  values  of  its  successors,  and  place  them 
in  the  proper  bins. 

(2)  Eliminate  from  further  consideration  the  node  whose 
successors  were  just  added. 

(3)  If  a  node  in  the  highest  non-empty  bin  is  a  term- 
inal node  in  the  tree,  stop.   Otherwise,  go  to  (1). 

In  each  case  when  a  node  is  placed  in  a  bin,  its  description 
is  placed  on  the  stack. 

Geist  [17]  has  done  extensive  computer  simulation  studies 
which  indicate  that  this  version  of  the  Jelinek  algorithm 
is,  in  most  cases  of  interest,  superior  to  the  Fano  algor- 
ithm [18]  as  long  as  sufficient  computer  memory  is  available. 
A  detailed  discussion  of  the  process  followed  in  the  decoder 
program  is  given  in  Appendix  A. 

B.   USE  OF  THE  DECODER  FOR  CONSTRUCTING  CODES 

As  was  stated  in  Chapter  II,  the  error-correcting  capa- 
bilities of  a  code  are  closely  related  to  the  respective 
weights  of  the  codewords.   Forney  [19]  has  given  a  method 
for  determining  the  codeword  weights  by  using  a  sequential 
decoding  simulator.   His  method  is  outlined  for  a  simulator 
using  the  Fano  algorithm  for  determining  the  codeword 
weights  of  fixed'  codes.   In  this  project  the  method  was 
suitably  modified  to  use  the  quantized  Jelinek  algorithm  for 
constructing  time-varying  codes  with  good  distance  properties. 
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Since  with  time-varying  codes,  free  distance  appears  to 
be  the  most  appropriate  distance  measure,  and  free  distance 
is  related  to  order-j  column  distance,  the  decoder  was  used 
to  measure  the  column  distance  of  successive  codewords. 
With  this  value  available,  an  attempt  was  then  made  to 
select  subsequent  generator  polynomials  with  the  aim  of 
maximizing  the  minimum  order-j  column  distance. 

The  steps  in  the  procedure  are  as  follows: 

(1)  Select  the  first  generator  polynomials,  Gi   ,  such 
that  d1  =  2.   Set  i  =  1,  j  =  2. 

Note:   k  =  1  for  systematic  codes. 

k  =  1,2  for  non-systematic  codes. 

(2)  Make  a  choice  for  G_jk). 

(3)  Using  these  generator  polynomials  and  the  decoder, 
search  to  level  j  in  the  tree. 

(4)  Compute  d..   If  d.  >  d,  ,  go  to  (7). 

J       J     J  -*- 

(5)  If  all  choices  for  G.  ^  are  exhausted,  go  to  (7). 

(6)  Make  another  choice  for  G .   .   Go  to  (3). 

(7)  If  i  =  T,  the  period  of  the  code,  stop.   Otherwise, 
j  =  J+l,  i  -    i+1,  and  go  to  (2). 

After  this  process  was  completed,  it  was  a  simple  matter 

to  modify  the  computer  program  to  test  the  code  thus 

generated. 
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IV.   CONSTRUCTION  OF  TIME-VARYING  CODES 

In  this  chapter,  the  several  methods  employed  in  an 
attempt  to  construct  good  time-varying  codes  will  be  dis- 
cussed.  Included  will  be  the  reasons  for  searching  for  a 
code  of  the  particular  class,  and  a  comparison  to  the  per- 
formance of  fixed  codes  of  the  same  memory.   The  distri- 
bution of  computations,  number  of  frames  erased  per  1000 
frames,  number  of  error  frames  per  1000  frames,  and  bit 
error  probability  are  compared  for  each  code  generated. 

All  codes  generated  were  of  rate  \   and  comparisons  were 
made  over  simulated  binary  symmetric  channels  with  cross- 
over probabilities  p  =  0.03125  and  p  =  0.04688.   For  each 
code,  an  attempt  was  made  to  decode  1000  frames  of  length 
256  bits  at  each  of  the  two  channel  probabilities.   Through' 
out  simulation,  the  all-zero  sequence  was  assumed  to  have 
been  transmitted.   Since  the  nominal  probability  of  a 
transmitted  symbol  being  a  zero  or  a  one  is  one-half,  no 
loss  of  generality  occurs,  but  this  affords  the  decoder  an 
unfair  advantage.   For  this  reason,  the  decoder  was  pro- 
grammed to  choose  the  "one-branch"  in  cases  where  ties 
occurred. 

A.   SYSTEMATIC  TIME-VARYING  CODES 
1.   Wagner  Codes 

Wagner  [12]  suggested  a  class  of  systematic, 
periodic,  binary  convolutional  codes  for  which  his  Gilbert 
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bound  applies.   The  encoder  is  depicted  in  Figure  5(a).   As 
before,  if  gi(u)  =  0,  no  wire  is  present,  and  if  g.(u)  =  1, 
a  wire  is  present.   The  generator  sequence,  gQ(u),...,g  (u) , 
is  formed  from  the  leftmost  (m+1)  bits  of  the  generator 
register  shown  in  Figure  5(b).   Each  time  unit,  the  infor- 
mation register  shifts  right  once  and  the  generator  register 
shifts  circularly  twice.   The  period  of  the  code  is 
T  =  3m+l  if  m  is  even  and  T  =  (3m+l)/2  if  m  is  odd. 

Using  the  fact  that  the  output  sequence  from  a 
k-stage  maximal  length  feedback  shift  register  is  periodic 
with  period  2  -1,  a  five-stage  MLFSR  was  used  to  generate 
codes  of  m  =  10  with  T  =  31.   A  complete  discussion  of  feed- 
back shift  registers  is  contained  in  Peterson  [20],  p.  147. 
To  generate  the  code,  coefficients  of  all  primitive  poly- 
nomials of  degree  five  were  tried  as  the  pattern  of  taps  on 
the  MLFSR.   The  code  generated  which  gave  the  best  perfor- 
mance is  compared  in  Table  II  to  the  performance  of  a  fixed 
systematic  code  constructed  by  Costello  [3]  for  high  free 
distance.   Costello's  code  was  constructed  for  m  =  35 
(36  bits)  and  taking  the  first  11  bits  to  form  a  memory  ten 
code  actually  resulted  in  a  generator  polynomial  with  m  =  9. 
The  truncated  version  (in  octal  form)  is: 

G  =  7324. 

Two  modifications  of  the  generated  Wagner  code  were 
tried  in  an  effort  to  improve  performance.   In  the  first 
modification,  called  "Wagner  (Mod  I)",  the  code  was  gener- 
ated as  before  with  the  exception  that  it  was  insured  that 
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(a).   Time-varying  Encoder 


Generator  Polynomial 
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(b).   Generator  Register. 


Figure  5-   Convo 


lutional  Encoder  for  Wagner  Codes 
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(a).   Errors  and  Erasures 


(b).   Frames  with  Number  of  Computations  >  N. 

Table  II.   Comparison  of  Wagner  Code  to  Fixed 
Systematic  Code  (m  =  10 ;. 
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the  present  information  bit  was  included  in  the  parity-check 
sequence  (i.e.,  gQ(u)  =  1  for  all  u).   The  results  compared 
to  Costello's  code  are  given  in  Table  III. 

This  modification  gave  only  a  small  improvement  over 
the  true  Wagner  code  with  the  bit  error  probability  at  a 
channel  error  probability  of  p  =  0.04688  being  slightly  less 
than  that  of  Costello's  code. 

In  the  second  modified  version,  the  memory  of  the 
encoder  was  Increased  to  12  and  g„ (u) , . . . ,g, ? (u)  were  gen- 
erated as  for  the  true  Wagner  code.   It  was  then  insured 
that  the  present  information  bit  and  one  past  information 
bit  were  included  in  the  parity-check  sequence  (i.e., 
gn(u)  =  g-i  (u)  =  1  for  all  u).   This  code  was  called  "Wagner 
(Mod  II)"  and  Table  IV  shows  a  comparison  of  results  obtained 
to  those  obtained  using  Costello's  code  with  m  =  12.   The 
fixed  code  generator  polynomial  (in  octal  form)  is: 

G  =  73244. 

As  expected,  this  code  was  an  improvement  over  the 
true  Wagner  code  since  the  memory  was  longer,  but  the  code 
did  not  show  improvement  over  a  comparable  fixed  code. 
2.   Shift  Register  Codes 

These  codes  were  generated  using  a  24-stage  maximal 
length  feedback  shift  register.   The  first,  called  "Code  A," 
was  formed  by  taking  as  the  generator  polynomials  each 
successive  24-bit  output  sequence  obtained  after  shifting 
the  register  right  two  bits.   The  second,  Called  "Code  B," 
was  formed  in  the  same  manner  except  that  the  output 
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sequences  were  taken  after  shifting  the  register  left  one 
bit.  For  performance  comparison,  Costello's  fixed  syste- 
matic code  with  m  =  23  was  used: 

G  =  732^6070. 
Again,  performing  the  truncation  on  the  m  =  35  code  yielded 
only  a  code  with  memory  m  =  20.   Code  A  was  chosen  since  the 
generator  polynomials  vary  like  those  of  a  Wagner  code 
although  the  period  of  the  code  is  much  greater  than 
(3m+l)/2.   Code  B  is  a  modification  which  was  tested  for 
comparison.   Results  are  given  in  Table  V. 
3.   high  Minimum  Column  Distance  Codes 

This  class  of  codes  was  generated  using  the  algorithm 
outlined  in  Section  II. B.   The  algorithm  is  a  modification 
of  Costello's  algorithm  Al  [3],  p.  H2»  ,  for  constructing 
rate  3g,  fixed,  systematic  codes  with  high  free  distance. 
The  difficulty  with  trying  to  construct  time-varying  codes 
in  this  manner  is  brought  about  by  the  fact  that  when 
deciding  upon  the  generator  polynomial,  G(u)  ,  none  of  the 
polynomials,  G(u+t),  1  <  t  <  T-u ,  are  known.   Therefore, 
one  cannot  measure  the  order-j  column  distance,  d^ ,  for 
j  >  u  to  get  an  estimate  of  the  magnitude  of  dfree  for  the 
polynomials  chosen  up  to  that  point.   The  codes  were  con- 
structed with  the  aim  of  maximizing  d.    for  each  step,  keep- 
ing in  mind  that  this  method  only  gives  high  minimum  column 
distance  measured  from  the  origin  node  and  that  the  minimum 
column  distance  measured  from  other  nodes  may  be  quite  low. 
The  codes  were  constructed  for  a  high  d^.   Comparison  to 
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(b).   Frames  with  Number  of  Computations  >  N. 

Table  V.   Comparison  of  Systematic  Shift  Register 
Codes  to  Fixed  Systematic  Code  (m  =  23; 


40 


fixed  code  performance  is  given  in  Table  VI.   The  codes  were 
generated  for  m  =  23  with  period  T  =  35 .   Costello's  code 
was  used  for  comparison. 

B.   NON-SYSTEMATIC  TIME-VARYING  CODES 

As  was  discussed  in  Chapter  II,  Costello  [31  derived  a 
lower  bound  on  free  distance  for  non-systematic,  periodic, 
convolutional  codes  which  suggests  that  there  exists  at 
least  one  code  of  this  class  with  decoding  properties 
offering  a  great  improvement  over  comparable  fixed  codes. 
For  this  reason,  a  search  for  members  of  this  class  of  codes 
was  conducted. 

1 .   Wagner-like  Codes 

Since  Wagner  [12]  was  the  first  to  offer  any  results 
concerning  periodic  codes,  a  non-systematic  version  of  his 
codes  was  tested.   Both  arrays  of  generator  polynomials 

ere  constructed  using  the  method  outlined  in  Section  IV. A. 1 
for  m  =  10.   The  performance  results  were  compared  to  a 
non-systematic,  high  free  distance,  fixed  code  constructed 
by  Costello  with  his  algorithm  A9  [31,  P-  1^0 .   Costello's 
code,  again,  was  constructed  for  m  =  35  and  the  generator 
polynomials  (in  octal  form),  truncated  to  m  =  10  are: 

G(1)  =  733^ 

G(2)  =  5334 
As  before,  truncation  actually  resulted  in  a  code  with  m=9- 
Performance  results  are  given  in  Table  VII. 


VI 
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(b).   Frames  with  Number  of  Computations  >  N 

Figure  VI.   Comparison  of  High  ^35  Code  to 
S         Fixed  Systematic  Code  (m  =  23). 
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2 .   Shift  Register  Codes 

This  m  =  23  code  was  generated  in  like  manner  to  the 
systematic  code  of  the  same  type  (Code  A)  with  the  exception 
that  a  different  24-stage  MLFSR  was  used  to  form  each  array 
of  generator  polynomials.   Comparison  of  performance  is  made 
in  Table  VIII  and  Figures  6  and  7  to  Costello's  non- 
systematic  fixed  code,  truncated  to  m  =  23.   The  generator 
polynomials  for  the  fixed  comparison  code  are: 

G(1)  =  73353367 

G(2)  =  53353367 

Since  no  errors  were  made  with  either  the  time-varying  code 
or  the  fixed  code,  the  dominant  parameter  for  comparison  is 
the  distribution  of  computations.   Therefore,  plots  of  the 
distribution  are  given  vice  tables.   As  was  stated  in 
Chapter  I,  the  computation  has  an  assymptotic  Pareto  dis- 
tribution of  the  form 

P  {C  >  N}  =  KN~P  . 

The  observed  distributions  for  these  codes  are  plotted  in 
Figures  6  and  7  on  a  log-log  scale.   Therefore,  the  curves 
should  approach  straight  lines  whose  slopes  are  the  negatives 
of  the  values  of  p  given  in  Table  I.   These  asymptotes  are 
displayed  in  the  figures. 

From  Figures  6  and  7  it  can  be  seen  that  ,  although 
the  shift  register  code  made  no  decoding  errors,  the  code 
caused  the  decoder  to  perform  many  more  computations  than 
it  did  with  the  fixed  code. 
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CODE 

ERROR 
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BIT  ERROR 
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Shift  Register 
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0 
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0 

0.0 

34 

Table  VIII.   Comparison  of  Non-systematic  Shift  Register  Codes 
to  Fixed  Non-systematic  Code  (m  =  23). 
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3.  High  Minimum  Column  Distance  Codes 
Non-systematic,  time-varying,  high  minimum  column 

distance  codes  were  next  investigated.   These  codes  were 

generated  using  the  algorithm  in  Section  II. B  to  construct 

two  arrays  of  generator  polynomials  with  a  high  d0(r.   The 

3d 

same  difficulties  noted  for  systematic  codes  of  this  type 
also  pertain  to  the  generation  of  non-systematic  codes. 
Again,  codes  of  m  =  23  with  period  T  =  35  were  generated 
and  Costello's  high  free  distance,  non-systematic  code 
truncated  to  m  =  23  was  used  for  comparison.   The  results 
are  given  in  Table  IX.   Plots  of  the  distribution  of  compu- 
tations are  given  in  Figures  8  and  9. 

Although  the  probability  of  a  decoding  error  is 
quite  low  with  this  time-varying  code,  the  number  of  com- 
putations performed  when  using  the  code  is  much  greater 
than  the  number  performed  when  using  a  comparable  fixed 
code  . 

4 .  Complementary  Codes 

Recently,  Bahl  and  Jelinek  [21]  have  investigated 
a  class  of  rate  h   fixed  convolut ional  codes  with  complemen- 
tary generators.   They  have  described  a  synthesis  and  a 
search  procedure  that  results  in  good  codes  up  to  memory 
m  =  23.   The  codes  constructed  come  close  to  achieving 
established  upper  bounds  on  free  distance  and  the  generator 
polynomials  satisfy  the  following  two  conditions: 

(1)  g(1)  =  g(2)  =  g(1)  =  g(2)  =  1 
K±J    fe0     fe0     6m     fem 

(2)  g{2)  =  g!;1^,      0.<  1  <  m, 
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(1)*  (1) 

where  g^     denotes  the  complement  of  g.   .   The  results 

of  this  report  [21]  suggested  a  search  for  periodic  codes 

with  complementary  generators. 

a.  High  Minimum  Column  Distance  Complementary  Codes 
Construction  of  codes  with  high  d-j-  was  again 

attempted  using  the  algorithm  of  Section  II. B  ensuring  that 
the  choices  made  for  the  generator  polynomials  satisfied 
the  two  stated  conditions  for  complementary  codes.   Codes 
with  m  =  23  and  period  T  =  35  were  generated.   Performance 
results  are  given  in  Table  X  with  a  comparison  to  the 
m  =  23  fixed  code  constructed  by  Bahl  and  Jelinek.   Plots 
of  the  distribution  of  computations  are  given  in  Figures 
10  and  11. 

b.  Alternating  Two  Complementary  Codes 

In  this  time-varying  code,  which  is  the  last 
type  investigated,  generator  arrays  were  formed  by  alter- 
nating two  fixed  complementary  codes  to  give  a  code  with 
period  T  =  2.   Since  the  fixed  complementary  codes  with 
m  =  23  used  for  comparison  make  no  decoding  errors,  it  was 
decided  to  use  shorter  memory  codes  which  do  cause  some 
decoding  errors  so  as  to  have  a  good  basis  for  comparison. 
Bahl  and  Jelinek  [21]  have  constructed  a  number  of  optimal 
complementary  codes  that  have  the  largest  attainable  free 
distance.   The  complete  list  of  this  set  for  2  <  m  <  8  is 
given  in  the  report.   Two  of  these  codes  with  m  =  6  were 
chosen  to  form  the  period  T  =  2  code.   The  four  generator 
polynomials  (in  octal  form)  are: 
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GJ1}  =  424      G_[2)    =  75^ 

G^1}  =  51^      G^2)  =  664 

Performance  results  for  the  two  fixed  codes  and  the  periodic 
code  are  given  in  Table  XI. 

As  can  be  seen  from  Table  XI,  the  performance 
of  the  periodic  code  comes  close  to  equaling  that  of  the 
fixed  codes  in  both  error  probability  and  distribution  of 
computations,  but  major  improvement  over  the  performance 
of  fixed  codes  is  not  exhibited. 
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V.   CONCLUSIONS 

In  all  types  of  time-varying  codes  investigated,  the 
results  obtained  did  not  show  improvement  aver  the  perfor- 
mance of  comparable  fixed  codes.   The  most  promising 
periodic  codes  appear  to  be  those  using  complementary 
generators  with  the  generator  arrays  formed  by  some  manner 
of  interleaving  optimum  fixed  code  polynomials.   Of  note 
here  is  the  fact  that  the  Bahl-Jelinek  fixed  convolutional 
codes  with  complementary  generators  exhibited  better  per- 
formance than  other  fixed  codes  with  longer  memory. 

Further  research  into  time-varying  codes  should  include 
a  study  of  the  distance  properties  of  good  codes  with  the 
aim  of  finding  a  systematic  method  to  generate  periodic 
codes  with  high  free  distance.   Experimental  methods  might 
entail  performing  perturbations  on  the  generator  arrays  of 
the  best  known  periodic  code  in  an  effort  to  improve  per- 
formance . 

Costello's  lower  bound  on  free  distance  for  rate  ^, 
non-systematic,  periodic  codes  [31  s  P-  81,  is  an  asymptotic 
bound  which  means  that  it  may  not  necessarily  apply  to 
codes  of  short  constraint  length.   This  fact  is  exemplified 
by  the  results  obtained  in  this  work  with  periodic  codes  up 
to  memory  m  =  23.   The  added  encoder/decoder  complexity 
required  to  generate  long  constraint  length  time-varying 
codes  is  not  practical  when  one  considers  that  equally 
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reliable  communications  can  be  achieved  using  less  complex, 
shorter  fixed  codes.   Time-varying  codes,  thus  far,  appear 
to  be  of  theoretical  significance  only. 
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APPENDIX  A 
DECODER  AND  ERROR  PATTERN  GENERATOR  PROGRAMS 

All  computer  simulation  was  accomplished  on  the  XDS-9300 
computer.   A  detailed  discussion  of  the  quantized  Jelinek 
decoder  and  the  error  pattern  generator  programs  will  now 
be  given. 

A.   DECODER  PROGRAM 

For  every  node  the  decoder  encounters  in  its  search 
through  the  tree,  three  Items  of  information  about  the  node 
must  be  saved  so  that  the  node  can  be  extended  if  it  becomes 
the  top  node  in  the  stack.   These  items  are  the  node  like- 
lihood value,  its  depth  into  the  tree,  and  the  encoder 
state  at  the  node.   The  encoder  state  is  necessary  so  that 
the  code  symbols  on  the  two  branches  emanating  from  the 
node  may  be  determined;  the  depth  is  necessary  so  that  these 
code  symbols  may  be  compared  with  the  corresponding  symbols 
in  the  received  sequence;  and  the  new  node  likelihood  values 
must  be  obtained  by  adding  the  branch  likelihood  values  of 
the  two  emanating  branches  to  the  extended  node  value. 

For  the  node  to  be  extended,  these  items  were  saved  in 
three  computer  storage  locations  with  the  symbolic  labels 
VALUE,  DEPTH,  and  STATE  respectively.   For  each  node  on  the 
stack,  six  consecutive  words  of  computer  memory  contained 
the  complete  node  description.   In  the  first  three  of  these 
were  stored  the  node  value,  depth,  and  encoder  state;  the 
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last  three  locations  contained  a  stack  pointer,  a  path 
pointer,  and  a  flag,  all  of  which  will  be  explained  later. 

During  simulation  it  was  convenient  to  have  all  positive 
node  values;  so  the  origin  node  was  biased  to  a  value  of 
1000  which  ensured  that  no  node  encountered  had  a  negative 
value . 

At  the  beginning  of  the  search  the  origin  node  is  the 
top  node.   Therefore,  DEPTH   and  STATE  are  initially  set  to 
zero,  and  VALUE  is  set  to  1000.   During  the  search,  the 
information  about  each  node  encountered  is  saved  either 
in  VALUE,  DEPTH,  and  STATE,  or  as  a  node  description  on  the 
stack  (or  both).   Since,  in  this  quantized  version  of  the 
algorithm,  node  descriptions  are  neither  rearranged  in  the 
stack  nor  physically  deleted  from  the  stack,  even  if  the 
node  reaches  the  top  of  the  stack  and  is  extended,  there 
are  several  node  descriptions  in  memory  which  are  no  longer 
on  the  stack.   Therefore,  a  method  is  necessary  whereby  the 
ordering  of  the  stack  contents  into  bins  may  be  accomplished 
and  node  descriptions  which  represent  nodes  still  on  the 
stack  may  be  determined.   The  aforementioned  stack  pointer 
and  two  arrays  called  bin  index  and  bin  inventory  are  used 
for  this  purpose.   For  each  bin,  the  bin  inventory  contains 
the  number  of  nodes  in  the  bin,  and  the  corresponding  loca- 
tion in  the  bin  index  contains  the  address  of  the  first 
word  of  the  node  description  for  one  of  the  nodes  in  the 
bin.   The  stack  pointer  for  this  node  contains  the  address 
of  the  first  word  of  the  node  description  for  another  node 
in  the  bin.   The  chain  continues  until  all  nodes  in  the  bin 
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are  linked  together  with  the  stack  pointer  of  the  node 
description  of  the  last  node  in  the  bin  being  empty. 

The  size  of  the  bin  index  and  bin  inventory  arrays  are 
determined  by  the  size  of  the  node  likelihood  value  quan- 
tizing increment  which  was  assigned  the  symbolic  label  H, 
and  which  was  chosen  so  as  to  satisfy  the  following  two 
conditions : 

(1)  H  is  at  least  as  great  as  the  maximum  positive 
branch  likelihood  value,  normalized  to  an  integer. 

(2)  H  is  restricted  to  be  a  power  of  two  (i.e.,  H  =  2  ) 
so  that  the  proper  bin  in  which  to  place  a  node  can 
be  determined  by  an  r-bit  shift  operation. 

To  find  the  value  for  H  which  satisfies  the  first  condition, 
consider  that  when  a  node  is  extended  and  the  two  branches 
are  compared  to  the  received  sequence,  there  are  three 
possible  outcomes:   (1)  a  branch  can  agree  with  the  received 
sequence  in  both  digits;  (2)  a  branch  can  disagree  with  the 
received  sequence  in  both  digits;  (3)  a  branch  can  disagree 
in  exactly  one  digit.   The  branch  likelihood  values  corres- 
pond to  the  branch  metric  values  in  Equation  (1.1),  nor- 
malized to  integer  values.   For  the  three  possible  outcomes, 
the  metric  values  at  each  of  the  two  channel  probabilities 
were  chosen  as  given  in  Table  XII.   From  the  information 
listed  in  Table  XII,  it  can  be  seen  that  a  quantizing  incre- 
ment which  satisfied  both  stated  conditions,  and  the  one 
used  during  simulation,  was  H  =  8. 

The  sequence  of  events  during  decoding  will  now  be 
described.   VALUE,  DEPTH,  and  STATE  are  set  to  the  respec- 
tive parameters  for  the  top  node  in  the  stack  and  the 
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Metric  Values 


p  =  0.03125      P  =  0.0*»6i 


Both  digits  agree  8  8 

Both  digits  disagree  -72  -64 

One  digit  disagrees  -32  -28 

Table  XII.   Metric  Values  Used  in  Simulation. 

branch  likelihood  values  for  the  two  branches  emanating 
from  that  node  are  computed.   Now  there  are  two  possible 
cases:   either  (1)  at  least  one  branch  value  is  positive,  or 
(2)  both  branch  values  are  negative.   The  two  cases  will  be 
considered  separately. 

Case  (1 ) .   If  at  least  one  branch  value  is  positive,  the 
successor  node  along  that  path  must  belong  in  either  the 
same  bin  as  the  extended  node  or  a  higher  bin  and,  there- 
fore, the  successor  node  must  be  the  new  top  node.   VALUE, 
DEPTH,  and  STATE  are  adjusted  to  the  corresponding  parameters 
for  the  new  node  and  the  node  description  for  the  other 
successor  must  be  stored  on  the  stack.   The  first  three 
items  are  computed  from  the  parameters  stored  In  VALUE, 
DEPTH,  and  STATE  along  with  the  calculated  branch  value. 
The  integer  part  of  the  successor  node  value  divided  by  H 
determines  the  proper  bin  in  which  the  node  belongs.   The 
bin  inventory  for  this  bin  is  increased  by  one,  and  the 
stack  pointer  of  the  new  node  description  is  set  to  the 
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address  which  is  stored  in  the  corresponding  location  of  the 
bin  index.   The  bin  index  is  then  set  to  the  address  of  the 
first  word  of  the  new  node  description.   The  path  pointer 
and  the  flag,  which  will  be  discussed  later,  are  used  to 
recover  the  information  symbols  on  the  chosen  path  after  a 
search  ends  on  a  terminal  node  in  the  tree.   They  are  now 
set  and  the  decoder  is  ready  to  extend  the  new  top  node 
using  the  present  parameters  in  VALUE,  DEPTH,  and  STATE. 

Case  (2 )  .   If  both  branch  values  are  negative,  it  is 
clear  that  there  may  be  nodes  on  the  stack  which  have 
greater  likelihood  values  than  the  two  successors,  and 
therefore  belong  in  higher  bins.   First  the  two  new  node 
descriptions  are  stored  on  the  stack.   Now,  since  the 
extended  node  was  in  the  highest  non-empty  bin,  the  decoder 
searches  down  the  bin  inventory  array,  starting  with  the 
bin  in  which  the  extended  node  was  located,  until  it  finds 
a  bin  with  non-zero  inventory.   When  this  bin  is  located, 
the  inventory  is  decreased  by  one,  VALUE,  DEPTH,  and  STATE 
are  set  to  the  respective  parameters  from  the  node  descrip- 
tion located  using  the  address  in  the  corresponding  bin  in- 
dex, and  the  bin  index  is  set  to  the  stack  pointer  address 
for  that  node.   The  decoder  is  now  ready  to  extend  the  new 
node  . 

As  was  stated  earlier,  each  frame  is  256  bits  long  which 
yields  a  256  branch  tree.   The  256  branches  are  followed  by 
an  m-branch  tail  corresponding  to  a  memory  span  of  zeros. 
Since  it  is  known  that  only  zeros  are  transmitted  in  the 
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tail,  the  decoder  considers  only  the  zero-branch  successor 
during  its  search  in  the  tail.   If  the  branch  value  is 
positive,  no  node  description  is  stored  and  the  successor 
node  is  extended.   If  the  branch  value  is  negative,  the 
successor  node  description  is  stored  on  the  stack  and  a  new 
top  node  is  located  in  the  same  manner  as  described  in 
Case  (2)  earlier. 

When  the  contents  of  DEPTH  Is  (256  +  m) ,  the  search  has 
ended  on  a  terminal  node  and  the  decoder  must  recover  the 
information  symbols  on  the  chosen  path.   The  last  two  items 
of  the  node  description  are  used  to  accomplish  this  function 
Had  a  description  of  every  node  encountered  been  stored, 
the  path  pointer  could  have  been  set  to  the  address  of  the 
description  of  its  predecessor  and  a  path  from  the  terminal 
node  to  the  origin  would  easily  have  been  determined;  but 
this  is  not  the  case,  since,  in  many  instances,  only  one 
node  description  is  stored.   The  path  pointer  is  set  to  the 
address  of  the  description  of  a  node's  predecessor  if  it  is 
stored,  or,  to  the  address  of  the  node  at  the  end  of  the 
second  branch  emanating  from  the  predecessor  if  it  is  not 
stored.   In  the  latter  case  the  flag  is  used  to  indicate 
whether  the  path  pointer  leads  to  the  predecessor  or  the 
second  branch.   The  decoder  can  then  follow  a  path  back  to 
the  origin  to  retrieve  the  information  symbols  along  the 
path  selected. 
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B.   ERROR  PATTERN  GENERATOR  PROGRAMS 

1.  Binary  Symmetric  Channel  (p  =  0.03125) 

The  binary  symmetric  channel  with  crossover  prob- 
ability 0.03125  =  1/32  was  simulated  using  a  23-stage 
maximal-length  feedback  shift  register.   The  periodic 
sequences  from  a  MLFSR  are  called  "pseudo-noise"  (or  p-n) 
sequences.   If  a  sequence  of  i  digits  (i  £  m)  is  picked  at 
random  from  the  output  sequence  of  an  m-stage  binary  MLFSR, 
then  each  non-zero  sequence  has  probability  2    /(2  -1)  and 
the  zero  sequence  has  probability  (2    -l)/(2  -1).   There- 
fore, with  a  23-stage  MLFSR,  an  all-zero  sequence  of  five 
digits  will  occur  in  the  output  with  an  approximate  prob- 
ability of  1/32.   Since  the  all-zero  sequence  is  easy  to 
detect  with  a  digital  machine,  and  since  it  was  assumed 
that  the  all-zero  string  of  information  bits  was  transmitted, 
the  error  pattern  generator  was  programmed  to  insert  an 
error  bit  (i.e.,  change  a  received  symbol  to  a  "1")  each 
time  the  five-digit  zero  sequence  occurred  in  the  output  of 
the  MLFSR.   This,  then,  happened  with  a  nominal  probability 
of  1/32. 

2 .  Binary  Symmetric  Channel  (p  =  0.0^688) 

This  channel  with  crossover  probability  0.0^688  =  3/6^ 
was  also  simulated  using  a  23-stage  MLFSR.   In  this  case, 
each  time  the  five-digit  zero  sequence  occurred,  the  error 
pattern  generator  was  programmed  to  insert  an  error  bit 
(probability  =  1/32).   If  the  five-digit  zero  sequence  did 
not  occur,  the  generator  then  checked  for  a  six-digit 
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sequence  of  zeros.   If  the  six-digit  sequence  of  zeros  was 
present  (probability  =  1/64)  the  error  bit  was  inserted. 
Therefore,  the  overall  probability  of  occurrence  of  an 
error  was  [(1/32)  +  (1/64)]  =  (3/64). 
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